Picture coding method and picture decoding method

ABSTRACT

The picture coding method of the present invention is a picture coding method for coding a picture on a block-by-block basis, comprising: a selection step of selecting one of at least two sizes as a size of a block on which orthogonal transformation should be performed; a transformation step of performing orthogonal transformation on a block having the selected size; a coding step of coding data of said block obtained in the transformation step; and a generation step of generating a coded stream that includes the coded data of the block and size information concerning the size selected in the selection step, wherein the size information indicates whether or not the size is a fixed block size within a predetermined section in the coded stream, and the predetermined section is one of a sequence, a group of pictures, a picture, a slice, and a macroblock.

TECHNICAL FIELD

The present invention relates to a picture coding method for efficientlycompressing a moving picture and a picture decoding method for decodingsuch compressed moving picture, as well as a picture coding apparatusand a picture decoding apparatus thereof, and more particularly to animproved technique for achieving high compression ratio.

BACKGROUND ART

In the age of multimedia that integrally handles audio, video and pixelvalues of others, existing information media, i.e. newspaper, magazine,television, radio, telephone and other means through which informationis conveyed to people, have recently come to be included in the scope ofmultimedia. In general, multimedia refers to representing not onlycharacters, but also graphics, voices, and especially pictures and thelike together in association with one another. However, in order toinclude the aforementioned existing information media in the scope ofmultimedia, it becomes absolutely necessary to represent suchinformation in digital form.

However, when calculating the amount of information contained in each ofthe aforementioned information media as the amount of digitalinformation, while the amount of information per character is 1˜2 bytesin the case of characters, the amount of information to be required is64 Kbits or over per second in the case of voices (telephone quality),and 100 Mbits or over per second in the case of a moving picture(current television reception quality). Thus, it is not realistic forthe aforementioned information media to handle such an enormous amountof information as it is in digital form. For example, although videophones are already in the actual use by use of Integrated ServicesDigital Network (ISDN) that offers a transmission speed of 64 Kbit/s˜1.5Mbit/s, it is not possible to transmit video of televisions and camerasdirectly through ISDN.

Against this backdrop, information compression techniques have becomerequired, and moving picture compression techniques compliant with H.261and H.263 standards recommended by ITU-T (InternationalTelecommunication Union-Telecommunication Standardization Sector) areemployed for video phones, for example. Moreover, according to aninformation compression technique compliant with the MPEG-1 standard, itis possible to store picture information into an ordinary music CD(compact disc) together with audio information.

Here, MPEG (Moving Picture Experts Group) is an international standardon compression of moving picture signals standardized by ISO/IEC(International Organization for Standardization/ InternationalElectrotechnical Commission), and MPEG-1 is a standard for compressingtelevision signal information approximately into one hundredth so that amoving picture signal can be transmitted at a rate of 1.5 Mbit/s.Furthermore, since a transmission speed achieved by the MPEG-1 standardis a middle-quality speed of about 1.5 Mbit/s, MPEG-2, which wasstandardized with a view to satisfying requirements for further improvedpicture quality, allows data transmission equivalent in quality totelevision broadcasting through which a moving picture signal istransmitted at a rate of 2˜15 Mbit/s. Moreover, MPEG-4 was standardizedby the working group (ISO/IEC JTC1/SC29/WG11) which promoted thestandardization of MPEG-1 and MPEG-2. MPEG-4, which provides a highercompression ratio than that of MPEG-1 and MPEG-2 and which enables anobject-based coding/decoding/operation, is capable of providing a newfunctionality required in this age of multimedia. At the beginning stageof standardization, MPEG-4 aimed at providing a low bit rate codingmethod, but it has been extended as a standard supporting more generalcoding that handles interlaced images as well as high bit rate coding.Currently, an effort has been made jointly by ISO/IEC and ITU-T forstandardizing MPEG-4 AVC and ITU-T H.264 as picture coding methods ofthe next generation that offer a higher compression ratio.

In general, in coding of a moving picture, the amount of information iscompressed by reducing redundancies in temporal and spatial directions.Therefore, in inter picture prediction coding aiming at reducingtemporal redundancies, motion estimation and the generation of apredicative image are carried out on a block-by-block basis withreference to forward or backward picture(s), and coding is thenperformed on the differential value between the obtained predictiveimage and an image in the current picture to be coded. Here, “picture”is a term denoting one image. In the case of a progressive image,“Picture” means a frame, whereas it means a frame or fields in the caseof an interlaced image. Here, “interlaced image” is an image of a framecomposed of two fields which are separated in capture time. In codingand decoding of an interlaced image, it is possible to handle one frameas (1) a frame as it is, (2) two fields, or (3) a frame structure or afield structure on a per-block basis within the frame.

A picture to be coded using intra picture prediction without referenceto any pictures shall be referred to as an I picture. A picture to becoded using inter picture prediction with reference to only one pictureshall be referred to as a P picture. And, a picture to be coded usinginter picture prediction with reference to two pictures at the same timeshall be referred to as a B picture. It is possible for a B picture torefer to two pictures which can be arbitrarily combined fromforward/backward pictures in display order. Reference images (referencepictures) can be determined for each block serving as a basiccoding/decoding unit. Distinction shall be made between such referencepictures by calling a reference picture to be described earlier in acoded bitstream as a first reference picture, and by calling a referencepicture to be described later in the bitstream as a second referencepicture. Note that as a condition for coding and decoding these types ofpictures, pictures used for reference are required to be already codedand decoded.

P pictures and B pictures are coded using motion compensated interpicture prediction. Coding by use of motion compensated inter pictureprediction is a coding method that employs motion compensation in interpicture prediction coding. Unlike a method for performing predictionsimply based on pixel values in a reference picture, motion estimationis a technique capable of improving prediction accuracy as well asreducing the amount of data by estimating the amount of motion(hereinafter referred to as “motion vector”) of each part within apicture and further by performing prediction in consideration of suchamount of motion. For example, it is possible to reduce the amount ofdata through motion compensation by estimating motion vectors of thecurrent picture to be coded and then by coding prediction residualsbetween prediction values obtained by shifting only the amount of therespective motion vectors and the current picture to be coded. In thistechnique, motion vectors are also recorded or transmitted in codedform, since motion vector information is required at the time ofdecoding.

Motion vectors are estimated on a per-macroblock basis. Morespecifically, a macroblock shall be previously fixed in the currentpicture to be coded, so as to estimate motion vectors by finding theposition of the most similar reference block of such macroblock withinthe search area in a reference picture.

FIG. 1 is a diagram illustrating an example data structure of abitstream. As FIG. 1 shows, the bitstream has a hierarchical structuresuch as below. The bitstream (Stream) is formed of more than one groupof pictures (GOP). By using GOPs as basic coding units, it becomespossible to edit a moving picture as well as to make a random access.Each GOP is made up of plural pictures, each of which is one of Ipicture, P picture, and B picture. Each picture is further made up ofplural slices. Each slice, which is a strip-shaped area within eachpicture, is made up of plural macroblocks. Moreover, each stream, GOP,picture, and slice includes a synchronization signal (sync) forindicating the ending point of each unit and a header (header) which isdata common to said each unit.

Note that when data is carried not in a bitstream that is a sequence ofstreams, but in a packet and the like that is a piecemeal unit, theheader and the data portion, which is the other part than the header,may be carried separately. In such case, the header and the data portionshall not be incorporated into the same bitstream, as shown in FIG. 1.In the case of a packet, however, even when the header and the dataportion are not transmitted contiguously, it is simply that the headercorresponding to the data portion is carried in another packet.Therefore, even when the header and the data portion are notincorporated into the same bitstream, the concept of a coded bitstreamdescribed with reference to FIG. 1 is also applicable to packets.

FIG. 2 is a block diagram showing the construction of an existingpicture coding apparatus. In this drawing, a picture coding apparatus 1is an apparatus for performing compression coding on an input picturesignal Vin, so as to output a coded picture signal Str which has beencoded into a bitstream by performing variable length coding and thelike. Such picture coding apparatus 1 is comprised of a motionestimation unit ME, a motion compensation unit MC, a subtraction unitSub, an orthogonal transformation unit T, a quantization unit Q, aninverse quantization unit IQ, an inverse orthogonal transformation unitIT, an addition unit Add, a picture memory PicMem, a switch SW, and avariable length coding unit VLC.

The picture signal Vin is inputted to the subtraction unit Sub and themotion estimation unit ME. The subtraction unit Sub calculates, as aprediction error, a difference between each image in the input picturesignal Vin and each predictive image on a block-by-block basis, andoutputs the calculated prediction error to the orthogonal transformationunit T. The orthogonal transformation unit T performs orthogonaltransformation on the prediction error to transform it into frequencycoefficients, and outputs such frequency coefficients to thequantization unit Q. The quantization unit Q quantizes such inputtedfrequency coefficients, and outputs the quantized values Qcoef to thevariable length coding unit VLC.

The inverse quantization unit IQ performs inverse quantization on thequantized values Qcoef so as to turn them into the frequencycoefficients, and outputs such frequency coefficients to the inverseorthogonal transformation unit IT. The inverse orthogonal transformationunit IT performs inverse frequency transformation on the frequencycoefficients so as to transform them into a prediction error, andoutputs such prediction error to the addition unit Add. The additionunit Add adds each prediction error and each predictive image outputtedfrom the motion estimation unit MC, so as to form a decoded image. Theswitch SW turns to ON when it is indicated that such decoded imageshould be stored, and such decoded image is to be stored into thepicture memory PicMem.

Meanwhile, the motion estimation unit ME, which receives the picturesignal Vin on a macroblock basis, detects an image area closest to suchinput image signal Vin from the among decoded pictures stored in thepicture memory PicMem, and determines motion vector(s) MV indicating theposition of such area. Motion vectors are estimated for each block,which is obtained by further dividing a macroblock. When this is done,it is possible to use more than one picture as reference pictures. Areference picture used for estimating a motion vector shall beidentified by an identification number (reference index Index). Thepicture numbers of the respective pictures stored in the picture memoryPicMem are associated with reference indices Index.

The motion compensation unit MC reads out an optimum picture as apredictive picture from among the decoded pictures stored in the picturememory PicMem, based on the motion vectors detected in the aboveprocessing and the reference indices Index.

The variable length coding unit VLC performs variable length coding oneach of the quantized values Qcoef, reference indices Index, and motionvectors MV so as to output them as a coded stream Str.

FIG. 3 is a block diagram showing the construction of an existingpicture decoding apparatus. In this drawing, units that operate in thesame manner as that of the units in the picture coding apparatus shownin FIG. 2 are assigned the same numbers, and descriptions thereof areomitted.

The variable length decoding unit VLD decodes the coded stream Str intoquantized values Qcoef, reference indices Index, and motion vectors MV.Those quantized values Qcoef, reference indices Index, and motionvectors MV are inputted into the picture memory PicMem, the motioncompensation unit MC, and the inverse quantization unit IQ, wheredecoding processing is performed. Processing to be performed in suchdecoding processing is equivalent to that performed in the existingpicture coding apparatus shown in FIG. 2.

(Non-patent document) ITU-T Rec. H.264|ISO/IEC 14496-10 AVC Joint FinalCommittee Draft of Joint Video Specification (2002-8-10).

However, according to the existing picture coding apparatus, it isdifficult to use a high-compression ratio to all images containing manypixels and to all images of a variety of contents. It is thus requiredfor such existing picture coding apparatus to be capable of improvingimage quality as well as offering a high compression ratio.

To be more specific, the existing picture coding apparatus uses a fixedsized block as a unit of performing orthogonal transformation(orthogonal transformation size). This makes it difficult to achieve ahigh compression ratio to a moving picture signal including pictureswith a variety of contents such as high- and low-resolution pictures aswell as pictures with many and few variations in brightness and colors.The reason is that an orthogonal transformation size is 8×8 pixels inthe case of MPEG-1, MPEG-2, and MPEG-4, for example, whereas anorthogonal transformation size is 4×4 pixels in the case of MPEG-4 AVC,i.e. ITU-T H.264. On that point, since pixels are more stronglycorrelated with one another and the density among pixels of a displaydevice (e.g. CRT) is higher compared with a low-resolution image, it isdeemed desirable to use a larger orthogonal transformation size for ahigh-resolution image (e.g. HDTV). Moreover, it is also desirable inmany cases that a larger orthogonal transformation size be used forcontent with a smaller number of high frequency components, whereas asmaller orthogonal transformation size be used for content with a largernumber of high frequency components.

DISCLOSURE OF INVENTION

The present invention has been conceived in view of the above problems,and it is an object of the present invention to provide a picture codingmethod, a picture decoding method, a picture coding apparatus, a picturedecoding apparatus, and a program thereof for performing coding at ahigh compression ratio, regardless of whether the resolution of picturesas well as the bit rate is high or low and whether there are manyvariations in brightness and colors.

In order to achieve the above object, the picture coding methodaccording to the present invention is a picture coding method for codinga picture on a block-by-block basis, comprising: a selection step ofselecting one of at least two sizes as a size of a block on whichorthogonal transformation should be performed; a transformation step ofperforming orthogonal transformation on a block having the selectedsize; a coding step of coding data of said block obtained in thetransformation step; and a generation step of generating a coded streamthat includes the coded data of the block and size informationconcerning the size selected in the selection step.

With this construction, since an orthogonal transformation size isselected, it is possible to perform orthogonal transformation in anappropriate manner according to whether the resolution is high or low,whether the bit rate is high or low, and whether there are manyvariations in brightness and colors. Accordingly, it becomes possible tocompress all sorts of pictures at a higher compression ratio.

Here, the size information may indicate whether or not the size is afixed block size within a predetermined section in the coded stream, andthe predetermined section may be one of a sequence, a group of pictures,a picture, a slice, and a macroblock.

With the above construction, it becomes possible to indicate whether ornot a block size for orthogonal transformation is fixed in the codedstream for every predetermined section described above.

Here, the size information may indicate whether the size selected in theselection step is one of a first size and a second size which is otherthan said first size, and the first size may be a size of a block madeup of 4 by 4 pixels, and the second size may be a size of a block madeup of 8 by 8 pixels.

With the above construction, it becomes possible to improve acompression ratio by selectively switching between the first size for a4×4 pixel block and the second size for an 8×8 pixel block.

Here, in the transformation step, the data of theorthogonally-transformed block may be quantized by use of a weightingmatrix corresponding to the size of said block.

With the above construction, a weighting matrix is used to change thesize of a quantization step on a frequency component basis at the timeof quantizing orthogonally-transformed frequency components by means ofweighting. By using a weighting matrix that corresponds to each of thefirst size and second size, it becomes possible to further improve acoding efficiency.

Here, in the coding step, the weighting matrix corresponding to thesecond size may be coded, and in the generation step, said codedweighting matrix may be incorporated into the coded stream.

With the above construction, it becomes possible to set the weightingmatrix corresponding to the first size as the default weighting matrixin both the picture coding and decoding methods, so as to enable theweighting matrix corresponding to the second size to be selectively usedin the picture coding method.

Here, in the selection step, one of the first size and the second sizemay be selected according to at least one of the following: a bit rateof the coded stream; a resolution of the picture; and an indication fromoutside.

With the above construction, it becomes possible to make a flexibleselection of either the first size or the second size according to anyone of the bit rate of the coded stream, the resolution of the picture,and an indication from outside.

As described above, according to the picture coding method and thepicture decoding method of the present invention, it becomes possible toefficiently compress any sorts of pictures regardless of whether thepicture resolution is high or low, whether the bit rate is high or low,and whether there are many variations in brightness and colors.

Furthermore, it is possible to achieve the above-described effectsthrough the picture coding apparatus, picture decoding apparatus, andprogram thereof according to the present invention.

As further information about the technical background to thisapplication, U.S. Provisional Patent Application Ser. No. 60/449,135filed on Feb. 21, 2003 is incorporated herein by reference.

BRIEF DESCRIPTION OF DRAWINGS

These and other objects, advantages and features of the invention willbecome apparent from the following description thereof taken inconjunction with the accompanying drawings that illustrate a specificembodiment of the invention. In the Drawings:

FIG. 1 is a diagram illustrating an example data structure of abitstream;

FIG. 2 is a block diagram showing a construction of an existing picturecoding apparatus;

FIG. 3 is a block diagram showing a construction of an existing picturedecoding apparatus;

FIG. 4 is a block diagram showing a construction of a picture codingapparatus according to an embodiment of the present invention;

FIGS. 5A˜5G are diagrams showing block sizes for motion compensation atthe time of inter picture coding;

FIG. 6A is a block diagram showing a 16×16 pixel luma macroblock and anorthogonal transformation block thereof;

FIG. 6B is a block diagram showing an 8×8 pixel chroma macroblock and anorthogonal transformation block thereof;

FIG. 7A is a diagram showing a weighting matrix for an 8×8 pixel block;

FIG. 7B is a diagram showing a weighting matrix for a 4×4 pixel block;

FIG. 8 is a flowchart showing coding processing performed in the picturecoding apparatus according to the present embodiment;

FIGS. 9A˜9C are diagrams showing where size information is inserted intoeach coded stream;

FIGS. 10A˜10C are diagrams showing where size information and aweighting matrix are inserted into each coded stream on a per-GOP basis;

FIGS. 11A˜11C are diagrams showing where size information and aweighting matrix are inserted into each coded stream on a per-slicebasis;

FIG. 12 is a flowchart showing an example variation of the picturecoding processing;

FIG. 13 is a flowchart showing picture coding processing to be performedin the case where a 4×4 weighting matrix is derived from an 8×8weighting matrix;

FIG. 14 is a block diagram showing a construction of a picture decodingapparatus according to the embodiment of the present invention;

FIG. 15 is a flowchart showing variable length decoding processingperformed by the picture decoding apparatus of the present embodiment;

FIG. 16 is a flowchart showing picture decoding processing to beperformed in the case where a weighting matrix W4mat is derived from aweighting matrix W8mat;

FIGS. 17A˜17C are diagrams illustrating a recording medium that stores aprogram for realizing the picture coding method and the picture decodingmethod according to the above embodiment by a computer system;

FIG. 18 is a block diagram showing an overall configuration of a contentsupply system;

FIG. 19 is a diagram showing an external view of a cellular phone thatemploys the picture coding method and the picture decoding method;

FIG. 20 is a block diagram showing a construction of the cellular phone;and

FIG. 21 is a diagram showing an overall configuration of a digitalbroadcasting system.

BEST MODE FOR CARRYING OUT THE INVENTION

The following describes an embodiment of the present invention withreference to FIGS. 4˜21.

<Picture Coding Apparatus>

FIG. 4 is a block diagram showing the construction of a picture codingapparatus according to the embodiment of the present invention. In thisdrawing, a picture coding apparatus 1 is comprised of a determinationunit DE, a motion estimation unit ME, a motion compensation unit MC, asubtraction unit Sub, an orthogonal transformation unit T1, aquantization unit Q1, an inverse quantization unit IQ1, an inverseorthogonal transformation unit IT1, an addition unit Add, a picturememory PicMem, a switch SW, and a variable length coding unit VLC1.

The determination unit DE determines (1) a block size Tsize of a blockon which orthogonal transformation should be performed by the orthogonaltransformation unit T1 (hereinafter referred to as “orthogonaltransformation size”) and (2) a weighting matrix Wmatrix for changing aquantization matrix to be used in the quantization unit Q1. There aretwo orthogonal transformation sizes Tsize, a first size and a secondsize, which are selectively determined on a per-macroblock basis. Here,the first size shall indicate 4×4 pixels and the second size shallindicate 8×8 pixels. As for weighting matrices Wmatrix, there are aweighting matrix W4 for the first size and a weighting matrix W8 for thesecond size.

An orthogonal transformation size is determined based on the resolutionof the current image to be coded, whether there are many variations inbrightness and colors (high frequency components) and the bit rate of acoded stream Str, and the like. For example, the determination unit DEselects an orthogonal transformation size in the following manner:

Example 1: the determination unit DE selects the second size when theresolution level of the current image to be coded is equal to or largerthan a first threshold, whereas it selects the first size when theresolution level of the current image is smaller than the firstthreshold. The resolution level of a HDTV and the like may be used asthe first threshold;

Example 2: the determination unit DE selects the second size when thebit rate after coding is equal to or larger than a second threshold,whereas it selects the first size when the bit rate is smaller than thesecond threshold. The second threshold may be 10 Mbps, for example; and

Example 3: the determination unit DE judges, from a previous result oforthogonal transformation, the number of high frequency components, andselects the second size when the number of high frequency components islarger than a third threshold, whereas the determination unit DE selectsthe first size when such number is smaller than the third threshold. Inother words, a judgment shall be made here on whether or not there aremany high frequency components, by counting the number or values offrequency coefficients which are higher than the “n”th order. Here, avalue which is predetermined according to picture quality and the likemay be the third threshold.

In the above examples 1˜3, the determination unit DE may furtherdetermine an orthogonal transformation size according to a block sizefor motion compensation.

FIGS. 5A˜5G are diagrams showing block sizes for motion compensation(hereinafter referred to as “motion compensation size(s)) at the time ofinter picture coding. Of them, FIG. 5A shows the motion compensationsize of 16×16 pixels, FIG. 5B shows 16×8 pixels, FIG. 5C shows 8×16pixels, and FIG. 5D shows 8×8 pixels. Furthermore, the motioncompensation size of 8×8 pixels shown in FIG. 5D may further be dividedinto one of the following sizes so as to be used as a motioncompensation size: 8×4 pixels shown in FIG. 5E, 4×8 pixels shown in FIG.5F, and 4×4 pixels shown in FIG. 5G. There is preferably a consistencybetween block sizes used as a motion compensation size and an orthogonaltransformation size so that processing can be simplified and artifactsat a block boundary will not be dispersed. In consideration of this, thedetermination unit DE determines an orthogonal transformation size in amanner in which an orthogonal transformation size becomes smaller than amotion compensation size.

More specifically, in the case of inter picture coding, when motioncompensation sizes are as shown in FIGS. 5A˜5D (16×16 pixels, 16×8pixels, 8×16 pixels, and 8×8 pixels), exactly one or more 8×8 pixelblocks shown in FIG. 5D is/are contained. In these cases, since higherefficiency is achieved by performing orthogonal transformation on an 8×8pixel block basis, the determination unit DE shall select the secondsize (8×8 pixels) as an orthogonal transformation size. Meanwhile, whena motion compensation size is smaller than 8×8 pixels as shown in FIGS.5E˜5G, the determination unit DE shall select the first size (4×4pixels).

The more frequently the determination unit DE switches orthogonaltransformation sizes between the first size and the second size, themore complicated the construction of the picture coding apparatus andpicture decoding apparatus becomes. A better coding efficiency, however,can be achieved by this.

Note that the determination unit DE may determine an orthogonaltransformation size on a slice/picture/GOP/sequence basis, instead of ona macroblock basis. Moreover, in the case where an orthogonaltransformation size is indicated from outside the picture codingapparatus, the determination unit DE may use such indicated size as anorthogonal transformation size Tsize.

The motion compensation unit MC reads out a reference block according tomotion vector(s) estimated by the motion estimation unit ME so as togenerate a predictive image from such reference block, and outputs thegenerated predictive image to the subtraction unit Sub. A predictiveimage is outputted for each block having a motion compensation sizeselected from among the motion compensation sizes shown in FIGS. 5A˜5G.

The subtraction unit Sub calculates, as a prediction error, thedifference between an image in the picture signal Vin and a predictiveimage for each block having a motion compensation size selected fromamong the motion compensation sizes shown in FIGS. 5A˜5G, and outputssuch calculated prediction error to the orthogonal transformation unitT1.

The orthogonal transformation unit T1 performs orthogonal transformationon a prediction error so as to transform it into frequency coefficientsfor each block having an orthogonal transformation size Tsize outputtedby the determination unit DE, and outputs the resultant to thequantization unit Q. Here, referring to FIGS. 6A and 6B, a descriptionis given of orthogonal transformation to be performed in the case wherean orthogonal transformation size Tsize is the second size (8×8 pixels).Illustrated in the left side of FIG. 6A is a 16×16 pixel lumamacroblock. Since the second size is indicated as an orthogonaltransformation size Tsize in the case of intra picture coding, theorthogonal transformation unit T1 performs orthogonal transformation onan 8×8 pixel block basis. Furthermore, as shown in the right side of theFIG. 6A, the orthogonal transformation unit T1 integrates four DCcomponents of four orthogonally-transformed 8×8 pixel blocks so as togenerate one 2×2 pixel luma DC block, and then performs 2×2 pixel blockorthogonal transformation on such luma DC block. Note that thegeneration of a luma DC block and the execution of 2×2 pixel blockorthogonal transformation may be omitted. Meanwhile, since the secondsize is indicated as an orthogonal transformation size Tsize in the caseof inter picture coding, the orthogonal transformation unit T1 performsorthogonal transformation on an 8×8 pixel block basis. In this case, thegeneration of a luma DC block and the execution of orthogonaltransformation shall not be carried out.

Illustrated in the left side of FIG. 6B is an 8×8 pixel chromamacroblock. Regarding a picture signal, in general, a chrominance signalhas half the number of pixels both horizontal and vertical directionscompared with a luminance signal. Thus, the number of pixels in a chromamacroblock is 8×8 pixels. In the case of intra picture coding, theorthogonal transformation unit T1 performs orthogonal transformation ona 4×4 pixel block basis, which is half the size of the second sizeindicated as an orthogonal transformation size Tsize. Then, theorthogonal transformation unit T1 integrates four DC components of fourorthogonally-transformed 4×4 chroma block so as to generate one 2×2pixel chroma DC block, and then performs 2×2 pixel block orthogonaltransformation on such chroma DC block. Note that the generation of achroma DC block and the execution of 2×2 pixel block orthogonaltransformation may be omitted. Meanwhile, in the case of inter picturecoding, the orthogonal transformation unit T1 performs orthogonaltransformation on a 4×4 pixel block basis, which is half the size of thesecond size indicated as an orthogonal transformation size Tsize. Inthis case, the generation of a chroma DC block and the execution oforthogonal transformation shall not be carried out.

As described above, when an orthogonal transformation size Tsizeindicates the second size, the orthogonal transformation unit T1performs two-step orthogonal transformation on a luma block and a chromablock in the case of intra picture coding, whereas it performs asingle-step orthogonal transformation in the case of inter picturecoding. Meanwhile, when an orthogonal transformation size Tsizeindicates the first size, the orthogonal transformation unit T1 performsorthogonal transformation as in the case of the second size since thedifference between the first and second sizes is the difference in ablock size, but a single-step coding shall be performed both in the caseof intra picture coding and inter picture coding.

The quantization unit Q1 quantizes a 4×4- or 8×8-pixel frequencycoefficient block inputted from the orthogonal transformation unit T1,and outputs quantized values Qcoef to the variable length coding unitVLC. In so doing, the quantization unit Q1 weights a quantization matrixaccording to a weighting matrix Wmatrix determined by the determinationunit DE, and quantizes the frequency coefficient block using suchweighted quantization matrix. FIGS. 7A and 7B are diagrams illustratingweighting matrices. When an orthogonal transformation size is the firstsize, a weighting matrix Wmatrix to be inputted from the determinationunit DE shall be a 4×4 pixel weighting matrix W4mat shown in FIG. 7B,whereas when an orthogonal transformation size is the second size, aweighting matrix Wmatrix shall be an 8×8 pixel weighting matrix W8matshown in FIG. 7A. By using a weighting matrix, the sizes of quantizationsteps are changed on a per-frequency component basis at the time ofquantizing orthogonally-transformed frequency components. It is known inMPEG-2 that the use of a weighting matrix improves coding efficiency.Since plural orthogonal transformation sizes are switched between themin the present embodiment, it becomes possible to further improve codingefficiency by employing a different weighting matrix for each of theorthogonal transformation sizes. In FIG. 7A, W8i,j (“i” and “j” are anyof 0˜7 respectively) indicates weighting to be applied to the componentat the “i”th row in the eight rows and the “j”th column in the eightcolumns. In FIG. 7B, W4i,j (“i” and “j” are any of 0˜3 respectively)indicates weighting to be applied to the component at the “i”th row inthe four rows and the “j”th column in the four columns.

The inverse quantization unit IQ1 performs inverse quantization on thequantized values Qcoef so as to turn them into frequency coefficients,and outputs such frequency coefficients to the inverse orthogonaltransformation unit IT1. Stated another way, the inverse quantizationunit IQ1 performs processing that is the inverse of the processingperformed by the quantization unit Q1, according to the weighting matrixWmatrix inputted from the determination unit DE.

The inverse orthogonal transformation unit IT1 performs inversefrequency transformation on the frequency coefficient block receivedfrom the inverse quantization unit IQ1 so as to transform it into aprediction error, and outputs such prediction error to the addition unitAdd. Stated another way, the inverse orthogonal transformation unit IT1performs processing that is the inverse of the processing performed bythe orthogonal transformation unit T1 for each block having anorthogonal transformation size Tsize inputted from the determinationunit DE.

The addition unit Add adds the prediction error and a predictive imageoutputted from the motion compensation unit MC for each block having thesame motion compensation size as the one on which the subtraction unitSub has performed processing, and generates a decoded image.

The switch SW turns to ON when it is indicated that such decoded imageshould be stored, and the decoded image is to be stored into the picturememory PicMem.

Meanwhile, the motion estimation unit ME, which receives the picturesignal Vin on a macroblock basis, estimates an image area closest tosuch input image signal Vin from among the decoded pictures stored inthe picture memory PicMem, and determines motion vector(s) MV indicatingthe position of such area. Motion vectors are estimated for each block,which is obtained by further dividing a macro block. When this is done,it is possible to use more than one picture as reference pictures. Areference picture used for estimating a motion vector shall be indicatedby an identification number (reference index Index). The picture numbersof the respective pictures stored in the picture memory PicMem areassociated with reference indices.

The motion compensation unit MC reads out an optimum picture as apredictive picture from among the decoded pictures stored in the picturememory PicMem, based on the motion vectors detected in the aboveprocessing and the reference indices Index.

The variable length coding unit VLC1 performs variable length coding oneach of the quantized values Qcoef, reference indices Index, and motionvectors MV so as to output them as a coded stream Str. In so doing, thevariable length coding unit VLC1 generates size information TsizeFlagaccording to each orthogonal transformation size Tsize, and codes suchsize information TsizeFlag and a weighting matrix Wmatrix. There are twotypes of weighting matrices Wmatrix: a weighting matrix W8mat for 8×8orthogonal transformation and a weighting matrix W4mat for 4×4orthogonal transformation.

FIG. 8 is a flowchart showing coding processing performed in the picturecoding apparatus according to the present embodiment. The determinationunit DE determines an orthogonal transformation size Tsize and aweighting matrix Wmatrix on a per-macroblock basis (S10). Orthogonaltransformation and quantization shall be performed based on suchdetermined orthogonal transformation size Tsize and weighting matrixWmatrix. Furthermore, the variable length coding unit VLC1 performscoding in a manner that allows size information TsizeFlag indicatingTsize to be incorporated into the coded stream Str for everypredetermined section, i.e. one of the following sections: sequence,GOP, picture, slice, and macroblock. Here, the size informationTsizeFlag means any one of the following: (A) all orthogonaltransformation sizes within a predetermined section are 8×8 pixels; (B)all orthogonal transformation sizes within a predetermined section are4×4 pixels; and (C) it is unknown whether all orthogonal transformationsizes within a predetermined section are either 8×8 pixels or 4×4 pixels(i.e. there is a mixture of 4×4 pixel blocks and 8×8 pixel blocks). Notethat it is not necessary to distinguish all the size informationTsizeFlag (A)˜(C) from one another, and therefore that a distinction maybe made, for example, only between (A) and (C), and (B) and (C).Furthermore, when there should not be a mixture of 8×8 and 4×4orthogonal transformation sizes, a distinction may be made only between(A) and (B).

As shown in FIG. 9A, when coding size information TsizeFlag on a per-GOPbasis as a predetermined section, the variable length coding unit VCL1judges whether all orthogonal transformation sizes Tsize within suchsection are either the second size (8×8 pixels) or the first size (4×4pixels) (S11 and S12).

When the result of the judgment indicates that there are only 8×8 pixelblocks, the variable length coding unit VLC1 codes size informationTsizeFlag indicating that all the orthogonal transformation blocks inthe GOP are sized 8×8 pixels (S13), and codes a weighting matrix W8matfor an 8×8 pixel block (S14). FIG. 10B shows a coded stream Str to begenerated in this case.

When the result of the judgment indicates that there are only 4×4 pixelblocks, the variable length coding unit VLC1 codes size informationTsizeFlag indicating that all the orthogonal transformation blocks inthe GOP are sized 4×4 pixels (S15), and codes a weighting matrix W4matfor a 4×4 pixel block (S16). FIG. 10C shows a coded stream Str to begenerated in this case.

When the result of the judgment indicates that all the orthogonaltransformation blocks cannot be sorted into either 8×8 pixel block or4×4 pixel block (i.e. there is a mixture of them), the variable lengthcoding unit VLC1 codes size information TsizeFlag indicating that allthe orthogonal transformation blocks in the GOP are not sized onlyeither 4×4 pixels or 8×8 pixels (S17), and codes a weighting matrixW4mat for a 4×4 pixel block and a weighting matrix W8mat for an 8×8pixel block (S18). FIG. 10A shows a coded stream Str to be generated inthis case. Note that in order to inform the decoding apparatus whetheran orthogonal transformation size Tsize of each macroblock is the firstsize or the second size, it is desirable for the variable length codingunit VLC1 to perform coding in a manner in which an orthogonaltransformation size Tsize will be incorporated on a macroblock basis.

In the above description, an example is given for the case where thevariable length coding unit VLC1 codes size information TsizeFlag on aper-GOP basis, but the variable length coding unit VLC1 may code sizeinformation TsizeFlag on a per-slice basis, as shown in FIG. 9B. In suchcase, weighting matrices shall be coded as shown in FIGS. 11A-11C.

FIG. 11A shows the structure of a coded stream in the case where thereis a mixture of the first size and the second size within apredetermined section. In the header of such coded stream, (prior to thecurrent picture to be coded) size information TsizeFlag indicating amixture of the above two orthogonal transformation sizes is coded, andan 8×8 orthogonal transformation weighting matrix W8mat and a 4×4orthogonal transformation weighting matrix W4mat are coded subsequentlyto the size information TsizeFlag. Note that the 8×8 orthogonaltransformation weighting matrix W8mat and the 4×4 orthogonaltransformation weighting matrix W4mat do not necessarily have to be inthis order, and therefore that these two types of matrices may bedetermined to be arranged in the inverse order. FIG. 11B shows thestructure of a coded stream in the case where only the second size isused for orthogonal transform within a predetermined section. In theheader of such coded stream, (prior to the current picture to be coded)size information TsizeFlag indicating the 8×8 orthogonal transformationsize is coded, and an 8×8 orthogonal transformation weighting matrixW8mat is coded subsequently to the size information TsizeFlag. FIG. 11Cshows the structure of a coded stream in the case where only the firstsize is used for orthogonal transform within a predetermined section. Inthe header of such coded stream, (prior to the current picture to becoded) size information TsizeFlag indicating the 4×4 orthogonaltransformation size is coded, and a 4×4 orthogonal transformationweighting matrix W4mat is coded subsequently to the size informationTsizeFlag.

Furthermore, as shown in FIG. 9C, the variable length coding unit VLC1may code size information TsizeFlag on a macroblock basis. In the caseof FIG. 9C, however, an orthogonal transformation size Tsize itself maybe coded, instead of size information TsizeFlag. The above-describedFIGS. 9A˜9C respectively indicate where size information TsizeFlag is tobe inserted. In the case where an orthogonal transformation size isswitched to another one for every stream or sequence made up of pluralpictures, it is desirable to insert size information TsizeFlag into thetop of each stream or sequence, as shown in FIG. 9A. This is effectivein the case where there are few changes in the size and quality ofpictures throughout a stream or a sequence. Meanwhile, in the case wherethe image contents change on a picture-by-picture basis and therefore anorthogonal transformation size needs to be switched to another one foreach picture, it is desirable to place size information TsizeFlag at thetop of each picture, as shown in FIG. 9B. Furthermore, in the case wherean orthogonal transformation size needs to be switched to another one ona macroblock basis, it is desirable to place size information TsizeFlagat the top of each macroblock, as shown in FIG. 9C.

<Variation>

(1) FIG. 12 is a flowchart showing an example variation of the picturecoding processing shown in FIG. 8. This drawing shows coding processingto be performed in the case where a distinction is made between theabove-described size information TsizeFlag (B) and (C). The same stepsas those shown in FIG. 8 are assigned the same numbers. With thisconstruction, size information TsizeFlag indicates whether or not allorthogonal transformation sizes in a predetermined section describedabove are 4×4 pixels, and such indication shall be represented by 1 bit.

(2) It is desirable that an optimum value is set to each weightingmatrix W8mat and weighting matrix W4mat. However, when the determinationunit DE determines a weighting matrix Wmatrix on a per-stream or-picture basis, it may be possible that a weighting matrix W8mat isderived from a weighting matrix W4mat, and vice versa.

FIG. 13 is a flowchart showing picture coding processing to be performedin the case where a weighting matrix W4mat is derived from a weightingmatrix W8mat. The flowchart is newly added with Step 19, compared withthe flowchart shown in FIG. 8. The same steps as those in FIG. 8 areassigned the same numbers, and descriptions thereof are omitted. Here,only different points between FIGS. 8 and 9 are described. In Step S19,the variable length coding unit VLC1 derives a 4×4 weighting matrixW4mat from an 8×8 weighting matrix W8mat in the following manner, andcodes the derived weighting matrix W4mat:

W4i,j=K*W8 2i, 2j (“i” and “j” are any of 0˜3 respectively, and “K” is aconstant number)

Note that the orthogonal transformation unit T1 derives a weightingmatrix W4mat in the similar manner, so as to use it for orthogonaltransformation.

Also note that the variable length coding unit VLC1 may not code thederived 4×4 weighting matrix W4mat.

Moreover, the quantization unit Q1 and the inverse quantization unit IQ1may use different weighting matrices for intra picture coding and interpicture coding, even when processing has been performed on the sameorthogonal transformation size in these types of coding. Furthermore,the quantization unit Q1 and the inverse quantization unit IQ1 may usedifferent weighting matrices for a luminance signal and a chrominancesignal.

Also, the quantization unit Q1 and the inverse quantization unit IQ1 mayselect a weighting matrix used for quantization from among pluralweighting matrices.

Furthermore, the quantization unit Q1 and the inverse quantization unitIQ1 may weight a quantization matrix using a weighting matrix only for alarger orthogonal transformation size (e.g. the second size) for whichthe use of a weighting matrix is especially effective, and may notweight a quantization matrix for the other size (the first size) using aweighting matrix.

<Picture Decoding Apparatus>

FIG. 14 is a block diagram showing the construction of a picturedecoding apparatus according to the embodiment of the present invention.

In this drawing, the same numbers are assigned to the same components asthose shown in the picture coding apparatus shown in FIG. 4, andtherefore descriptions thereof are omitted.

The coded stream Str is decoded by the variable length decoding unitVLD1. The resultant of such decoding includes weighting matrices Wmatrixand orthogonal transformation sizes Tsize. The inverse quantization unitIQ1 weights a quantization matrix using a weighting matrix Wmatrix, andperforms inverse quantization using the weighted quantization matrix.The inverse orthogonal transformation unit IT1 performs inverseorthogonal transformation in accordance with a determined orthogonaltransformation size Tsize. The other operations are the same as thosedescribed in the block diagram of FIG. 3 that shows the picture decodingapparatus realizing an existing picture decoding method.

FIG. 15 is a flowchart showing variable length decoding processingperformed by the picture decoding apparatus of the present embodiment.As shown in this drawing, the variable length decoding unit VLD1 decodessize information TsizeFlag incorporated in the coded stream Str (S30),and judges whether such size information TsizeFlag indicates that thereare only blocks having the second size (8×8 pixels) (S31). When judgingthat there are only blocks having the second size, the variable lengthdecoding unit VLD1 decodes a weighting matrix W8mat (S33). Meanwhile,when the size information TsizeFlag does not indicate that only 8×8orthogonal transformation has been applied and when such sizeinformation TsizeFlag indicates that there are only blocks having thefirst size (4×4 pixels) (S32), the variable length decoding unit VLD1decodes a 4×4 weighting matrix W4mat (S34). Furthermore, when the sizeinformation TsizeFlag does not indicate that only 4×4 orthogonaltransformation has been applied, the variable length decoding unit VLD1decodes an 8×8 weighting matrix W8mat and a 4×4 weighting matrix W4mat(S35). Inverse quantization and inverse orthogonal transformation arethen performed according to the result of this decoding.

Note that when there is no weighting matrix W4mat coded in the codedstream Str, the picture decoding apparatus may derive a weighting matrixW4mat from a weighting matrix W8mat. FIG. 16 is a flowchart showingpicture decoding processing to be performed in the case where aweighting matrix W4mat is derived from a weighting matrix W8mat. Thisflowchart is newly added with Step S36, compared with FIG. 15. In StepS36, a weighting matrix W4mat is derived from a weighting matrix W8mat,in the manner equivalent to that of Step S19 shown in FIG. 13.

As described above, the picture decoding apparatus according to thepresent embodiment is capable of correctly decoding the coded stream Strwhich has been coded by the picture coding apparatus shown in FIG. 4.Note that an orthogonal transformation size Tsize is described above asbeing obtained from the variable length decoding unit VLD1. However, inthe case where an orthogonal transformation size Tsize is switched toanother one depending on the size of a motion compensation block asdescribed above, an orthogonal transformation size Tsize may be derivedbased on the size of a motion compensation block.

Furthermore, if a program for realizing the picture coding method andthe picture decoding method as shown in the aforementioned embodimentare recorded on a recording medium such as a flexible disk, it becomespossible to easily perform the processing presented in the aboveembodiment in an independent computer system.

FIGS. 17A, 17B, and 17C are diagrams illustrating a recording mediumthat stores a program for realizing the picture coding method and thepicture decoding method according to the aforementioned embodiment.

FIG. 17B shows an external view of a flexible disk viewed from thefront, its schematic cross-sectional view, and the flexible disk itself,while FIG. 17A illustrates an example physical format of the flexibledisk as a recording medium itself. The flexible disk FD is contained ina case F, and a plurality of tracks Tr are formed concentrically on thesurface of the flexible disk FD in the radius direction from theperiphery, each track being divided into 16 sectors Se in the angulardirection. Therefore, in the flexible disk storing the above-mentionedprogram, the picture coding method and the picture decoding method assuch program is recorded in an area allocated for it on the flexibledisk FD.

Meanwhile, FIG. 17C shows the structure for recording and reading outthe program on and from the flexible disk FD. When the program realizingthe above picture coding method and picture decoding method is to berecorded onto the flexible disk FD, such program shall be written by theuse of the computer system Cs via a flexible disk drive FDD. Meanwhile,when the picture coding method and the picture decoding method are to beconstructed in the computer system Cs through the program on theflexible disk FD, the program shall be read out from the flexible diskFD via the flexible disk drive FDD and then transferred to the computersystem Cs.

The above description is given on the assumption that a recording mediumis a flexible disk, but an optical disc may also be used. In addition,the recording medium is not limited to this, and any other medium suchas an IC card and a ROM cassette capable of recording a program can alsobe used.

The following describes application examples of the picture codingmethod and picture decoding method as shown in the above embodiment aswell as a system using them.

FIG. 18 is a block diagram showing an overall configuration of a contentsupply system ex100 that realizes a content distribution service. Thearea for providing a communication service is divided into cells ofdesired size, and base stations ex107˜ex110, which are fixed wirelessstations, are placed in the respective cells.

In this content supply system ex100, devices such as a computer ex111, aPDA (Personal Digital Assistant) ex112, a camera ex113, a cellular phoneex114, and a camera-equipped cellular phone ex115 are respectivelyconnected to the Internet ex101 via an Internet service provider ex102,a telephone network ex104, and the base stations ex107˜ex110.

However, the content supply system ex100 is not limited to thecombination as shown in FIG. 18, and may be connected to a combinationof any of them. Also, each of the devices may be connected directly tothe telephone network ex104, not via the base stations ex107˜ex110,which are fixed wireless stations.

The camera ex113 is a device such as a digital video camera capable ofshooting moving pictures. The cellular phone may be a cellular phone ofa PDC (Personal Digital Communication) system, a CDMA (Code DivisionMultiple Access) system, a W-CDMA (Wideband-Code Division MultipleAccess) system or a GSM (Global System for Mobile Communications)system, a PHS (Personal Handyphone system) or the like, and may be anyone of these.

Furthermore, a streaming server ex103 is connected to the camera ex113via the base station ex109 and the telephone network ex104, whichenables live distribution or the like based on coded data transmitted bythe user using the camera ex113. Either the camera ex113 or a server andthe like capable of data transmission processing may code the shot data.Also, moving picture data shot by a camera ex116 may be transmitted tothe streaming server ex103 via the computer ex111. The camera ex116 is adevice such as a digital camera capable of shooting still pictures andmoving pictures. In this case, either the camera ex116 or the computerex111 may code the moving picture data. In this case, an LSI ex117included in the computer ex111 or the camera ex116 performs codingprocessing. Note that software for coding and decoding may be integratedinto a certain type of storage medium (such as a CD-ROM, a flexible diskand a hard disk) that is a recording medium readable by the computerex111 and the like. Furthermore, the camera-equipped cellular phoneex115 may transmit the moving picture data. This moving picture data isdata coded by an LSI included in the cellular phone ex115.

In this content supply system ex100, content (e.g. a music live video)which has been shot by the user using the camera ex113, the camera ex116or the like is coded in the same manner as the above-describedembodiment and transmitted to the streaming server ex103, and thestreaming server ex103 makes stream distribution of the content data toclients at their request. The clients here include the computer ex111,the PDA ex112, the camera ex113, the cellular phone ex114 and so forthcapable of decoding the above coded data. The content supply systemex100 with the above configuration is a system that enables the clientsto receive and reproduce the coded data and realizes personalbroadcasting by allowing them to receive, decode and reproduce the datain real time.

The picture coding apparatus and picture decoding apparatus presented inthe above embodiment can be used for coding and decoding to be performedin each of the devices making up the above system.

An explanation is given of a cellular phone as an example.

FIG. 19 is a diagram showing the cellular phone ex115 that employs thepicture coding method and the picture decoding method explained in theabove embodiment. The cellular phone ex115 has an antenna ex201 fortransmitting/receiving radio waves to and from the base station ex110, acamera unit ex203 such as a CCD camera capable of shooting video andstill pictures, a display unit ex202 such as a liquid crystal displayfor displaying the data obtained by decoding video and the like shot bythe camera unit ex203 and video and the like received by the antennaex201, a main body equipped with a group of operation keys ex204, avoice output unit ex208 such as a speaker for outputting voices, a voiceinput unit ex205 such as a microphone for inputting voices, a recordingmedium ex207 for storing coded data or decoded data such as data ofmoving pictures or still pictures shot by the camera, data of receivede-mails and moving picture data or still picture data, and a slot unitex206 for enabling the recording medium ex207 to be attached to thecellular phone ex115. The recording medium ex207 is embodied as a flashmemory element, a kind of EEPROM (Electrically Erasable and ProgrammableRead Only Memory) that is an electrically erasable and rewritablenonvolatile memory, stored in a plastic case such as an SD card.

Next, referring to FIG. 20, a description is given of the cellular phoneex115. In the cellular phone ex115, a main control unit ex311 forcentrally controlling the display unit ex202 and each unit of the mainbody having the operation keys ex204 is configured in a manner in whicha power supply circuit unit ex310, an operation input control unitex304, a picture coding unit ex312, a camera interface unit ex303, anLCD (Liquid Crystal Display) control unit ex302, a picture decoding unitex309, a multiplexing/demultiplexing unit ex308, a recording/reproducingunit ex307, a modem circuit unit ex306, and a voice processing unitex305 are interconnected via a synchronous bus ex313.

When a call-end key or a power key is turned on by a user operation, thepower supply circuit unit ex310 supplies each unit with power from abattery pack, and activates the camera-equipped digital cellular phoneex115 to make it into a ready state.

In the cellular phone ex115, the voice processing unit ex305 converts avoice signal received by the voice input unit ex205 in conversation modeinto digital voice data under the control of the main control unit ex311comprised of a CPU, a ROM, a RAM and others, the modem circuit unitex306 performs spread spectrum processing on it, and a transmit/receivecircuit unit ex301 performs digital-to-analog conversion processing andfrequency transformation processing on the data, so as to transmit theresultant via the antenna ex201. Also, in the cellular phone ex115, datareceived by the antenna ex201 in conversation mode is amplified andperformed of frequency transformation processing and analog-to-digitalconversion processing, the modem circuit unit ex306 performs inversespread spectrum processing on the resultant, and the voice processingunit ex305 converts it into analog voice data, so as to output it viathe voice output unit ex208.

Furthermore, when sending an e-mail in data communication mode, textdata of the e-mail inputted by operating the operation keys ex204 on themain body is sent out to the main control unit ex311 via the operationinput control unit ex304. In the main control unit ex311, after themodem circuit unit ex306 performs spread spectrum processing on the textdata and the transmit/receive circuit unit ex301 performsdigital-to-analog conversion processing and frequency transformationprocessing on it, the resultant is transmitted to the base station ex110via the antenna ex201.

When picture data is transmitted in data communication mode, the picturedata shot by the camera unit ex203 is supplied to the picture codingunit ex312 via the camera interface unit ex303. When picture data is notto be transmitted, it is also possible to display such picture data shotby the camera unit ex203 directly on the display unit ex202 via thecamera interface unit ex303 and the LCD control unit ex302.

The picture coding unit ex312, which includes the picture codingapparatus according to the present invention, performs compressioncoding on the picture data supplied from the camera unit ex203 using thecoding method employed by the picture coding apparatus presented in theabove embodiment, so as to convert it into coded picture data, and sendsit out to the multiplexing/demultiplexing unit ex308. At this time, thecellular phone ex115 sends voices received by the voice input unit ex205while the shooting by the camera unit ex203 is taking place, to themultiplexing/demultiplexing unit ex308 as digital voice data via thevoice processing unit ex305.

The multiplexing/demultiplexing unit ex308 multiplexes the coded picturedata supplied from the picture coding unit ex312 and the voice datasupplied from the voice processing unit ex305 using a predeterminedmethod, the modem circuit unit ex306 performs spread spectrum processingon the resulting multiplexed data, and the transmit/receive circuit unitex301 performs digital-to-analog conversion processing and frequencytransformation processing on the resultant, so as to transmit theprocessed data via the antenna ex201.

When receiving, in data communication mode, moving picture file datawhich is linked to a Web page or the like, the modem circuit unit ex306performs inverse spread spectrum processing on the received signalreceived from the base station ex110 via the antenna ex201, and sendsout the resulting multiplexed data to the multiplexing/demultiplexingunit ex308.

In order to decode the multiplexed data received via the antenna ex201,the multiplexing/demultiplexing unit ex308 separates the multiplexeddata into a coded bitstream of picture data and a coded bitstream ofvoice data, and supplies such coded picture data to the picture decodingunit ex309 and such voice data to the voice processing unit ex305 viathe synchronous bus ex313.

Next, the picture decoding unit ex309, which includes the picturedecoding apparatus according to the present invention, decodes the codedbitstream of the picture data using the decoding method paired with thecoding method shown in the above-mentioned embodiment so as to generatemoving picture data for reproduction, and supplies such data to thedisplay unit ex202 via the LCD control unit ex302. Accordingly, movingpicture data included in the moving picture file linked to a Web page,for instance, is displayed. At the same time, the voice processing unitex305 converts the voice data into an analog voice signal, and thensupplies this to the voice output unit ex208. Accordingly, voice dataincluded in the moving picture file linked to a Web page, for instance,is reproduced.

Note that the aforementioned system is not an exclusive example andtherefore that at least either the picture coding apparatus or thepicture decoding apparatus of the above embodiment can be incorporatedinto a digital broadcasting system as shown in FIG. 21, against thebackdrop that satellite/terrestrial digital broadcasting has been arecent topic of conversation. To be more specific, at a broadcastingstation ex409, a coded bitstream of video information is transmitted, byradio waves, to a satellite ex410 for communications or broadcasting.Upon receipt of it, the broadcast satellite ex410 transmits radio wavesfor broadcasting, an antenna ex406 of a house equipped with satellitebroadcasting reception facilities receives such radio waves, and anapparatus such as a television (receiver) ex401 and a set top box (STP)ex407 decodes the coded bitstream and reproduces the decoded data. Thepicture decoding apparatus as shown in the above-mentioned embodimentcan be implemented in the reproduction apparatus ex403 for reading anddecoding the coded bitstream recorded on a storage medium ex402 that isa recording medium such as a CD and a DVD. In this case, a reproducedvideo signal is displayed on a monitor ex404. It is also conceivablethat the picture decoding apparatus is implemented in the set top boxex407 connected to a cable ex405 for cable television or the antennaex406 for satellite/terrestrial broadcasting so as to reproduce it on atelevision monitor ex408. In this case, the picture decoding apparatusmay be incorporated into the television, not in the set top box. Or, acar ex412 with an antenna ex411 can receive a signal from the satelliteex410, the base station ex107 or the like, so as to reproduce a movingpicture on a display device such as a car navigation system ex413mounted on the car ex412.

Furthermore, it is also possible to code a picture signal by the picturecoding apparatus presented in the above embodiment and to record theresultant in a recording medium. Examples include a DVD recorder forrecording a picture signal on a DVD disc ex421 and a recorder ex420 suchas a disc recorder for recording a picture signal on a hard disk.Moreover, a picture signal can also be recorded in an SD card ex422. Ifthe recorder ex420 is equipped with the picture decoding apparatuspresented in the above embodiment, it is possible to reproduce a picturesignal recorded on the DVD disc ex421 or in the SD card ex422, anddisplay it on the monitor ex408.

As the configuration of the car navigation system ex413, theconfiguration without the camera unit ex203 and the camera interfaceunit ex303, out of the configuration shown in FIG. 20, is conceivable.The same is applicable to the computer ex111, the television (receiver)ex401 and the like.

Concerning the terminals such as the cellular phone ex114, atransmitting/receiving terminal having both an encoder and a decoder, aswell as a transmitting terminal only with an encoder, and a receivingterminal only with a decoder are possible as forms of implementation.

As stated above, it is possible to employ the picture coding method andthe picture decoding method presented in the above embodiment into anyone of the above-described devices and systems. Accordingly, it becomespossible to achieve an effect described in the aforementionedembodiment.

From the invention thus described, it will be obvious that theembodiment of the invention may be varied in many ways. Such variationsare not to be regarded as a departure from the spirit and scope of theinvention, and all such modifications as would be obvious to one skilledin the art are intended for inclusion within the scope of the followingclaims.

INDUSTRIAL APPLICABILITY

The present invention is suited for use as a picture coding apparatusfor coding a picture on a block-by-block basis and a picture decodingapparatus, as well as a web server for distributing a moving picture, anetwork terminal for receiving such moving picture, a digital cameracapable of recording and reproduction a moving picture, acamera-equipped cellular phone, a DVD recorder/player, a PDA, a personalcomputer, and the like.

1.-26. (canceled)
 27. A coding method for coding image data on ablock-by-block basis, comprising: selecting, as orthogonaltransformation sizes of all blocks included in a predetermined imagedata unit on which orthogonal transformation should be performed, afirst size or a mixture of the first size and a second size, the firstsize being smaller than the second size; orthogonal transforming ablock, from all blocks included in the predetermined image data unit,having the selected orthogonal transformation size to obtain transformcoefficients; quantizing the transform coefficients using a quantizationmatrix corresponding to the selected orthogonal transformation size;variable length coding the quantized transform coefficients to obtain acoded stream; and inserting, into the coded stream, size informationindicating whether orthogonal transformation sizes of all blocksincluded in the predetermined image data unit are the first size or themixture of the first size and the second size; wherein, in the casewhere the mixture of the first size and the second size is selected anda motion compensation block size, which is used to perform motioncompensation on the block on which orthogonal transformation should beperformed, is smaller than the second size, a subsequent orthogonaltransforming is performed using only the first size as the orthogonaltransformation size.
 28. The coding method according to claim 27,wherein the size information is placed in each picture in the codedstream.
 29. The coding method according to claim 27, wherein the firstsize indicates a size of a block which is 4 pixels high and 4 pixelswide, and the second size indicates a size of a block which is 8 pixelshigh and 8 pixels wide.